When image data is transmitted from a transmission device to a reception device or when image data is stored in a storage device, for example, highly-efficiency encoding of the image data is performed due to the high data amount of the image data. Here, “high-efficiency encoding” represents encoding processing in which a certain data sequence is converted into another data sequence, that is, processing to compress the data amount of the data sequence. Especially, moving image data is often subject to high-efficiency encoding.
The H.264/AVC standard is widely used as typical moving image encoding algorithm, for example. The H.264/AVC standard is encoding algorithm which is defined in H.264 of International Telecommunication Union Telecommunication Standardization Sector (ITU-T). The H.264/AVC standard is defined in Moving Picture Experts Group phase 4 Advanced Video Coding (MPEG-4 AVC) of International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC), as well.
Examples of moving picture data mainly include data of which a constituent unit is only a flame and data of which a constituent unit is a field. A constituent unit of moving image data is called a picture. As moving image encoding, there is a case where one picture is divided into a plurality of blocks so as to form a slice including a plurality of blocks. One block is composed of n×m pieces of pixels and one picture is divided into one or more slices. n and m are integers which are 4 or larger and thus, a block of 16×16 pixels is often used. In the H.264/AVC standard, this block is called a macro block.
Prediction encoding is widely used as moving image encoding algorithm. Intra-picture prediction (intra prediction) coding is used as typical prediction encoding. Intra prediction encoding utilizes high correlativity of moving image data in a spatial direction, so that data of other coded pictures are not used. Accordingly, regarding a picture which has undergone intra prediction encoding, an original image may be restored only by information in the picture.
Further, prediction is performed by using high correlativity of moving image data in a temporal direction, in inter-picture prediction (inter prediction) coding. Moving image data often has a high degree of similarity between picture data on certain timing and picture data on the following timing, so that this property is used in inter prediction encoding.
In inter prediction encoding, an encoding object picture which is an original image is divided into a plurality of blocks and a decode image of an encoded reference picture is referred in a block unit so as to select a region similar to an encoding object block through block matching. Subsequently, a difference between pixel values of the similar region and the encoding object block (pixel difference) is obtained and redundancy is removed. Then, information of a motion vector indicating the similar region and the pixel difference from which redundancy is removed are encoded, realizing high-efficiency encoding with a high compression rate.
For example, in a data transmission system which employs inter prediction encoding, a transmission device encodes information of a motion vector which indicates a “motion” from a previous picture to an encoding object picture. Further, the transmission device encodes a pixel difference between a prediction image, which is generated from the previous picture by using the motion vector, of the encoding object picture and an original image of the encoding object picture. Then, the transmission device transmits the encoded information of the motion vector and the encoded pixel difference to a reception device. On the other hand, the reception device restores the original image of the encoding object picture on the basis of the information of the motion vector and the pixel difference that are received.
A moving image encoding device which is employed in such transmission device or the like transmits a picture which is subjected to intra prediction encoding in a regular cycle by using the group of pictures (GOP) configuration and performs inter prediction encoding with respect to the rest of pictures to transmit the pictures. In the GOP configuration, three kinds of pictures which are an I picture, a P picture, and a B picture which correspond to prediction algorithm are defined. An I picture is a picture which is encoded through intra prediction encoding which enables restoration of an original image only with information in the picture without referring to other pictures. A P picture and a B picture are pictures which are encoded mainly through inter prediction encoding.
A P picture is a picture which is obtained by performing forward inter-picture prediction from past reference pictures and encoding a pixel difference (prediction error). A B picture is a picture which is obtained by performing bidirectional inter-picture prediction from past and future reference pictures and encoding a prediction error. A future picture is used for prediction to obtain a B picture, so that a future reference picture which is to be used for prediction is encoded before encoding of the B picture and decoded.
Inter prediction encoding is now described in detail. Taking the H.264/AVC standard as an example, a macro block is divided into a plurality of sub blocks which are smaller than the macro block and a sub block or a macro block is set as an encoding object block. There are various inter prediction modes depending on a shape of a sub block. In inter prediction, reference indexes are assigned for a plurality of decode images which are reference destinations and a reference index, a motion vector, and a pixel difference are obtained in a sub block unit.
As information, which is to be encoded, of a motion vector, a difference vector between a prediction vector, as which a motion vector of a peripheral macro block is used, and a motion vector which is obtained by block matching is used. This difference vector is decreased as much as possible, being able to enhance encoding efficiency.
In the H.264/AVC standard, a skip encoding mode which is an implicit inter prediction mode is used so as to further enhance a compression rate. In the skip encoding mode, it is possible to decode an original image without transmitting information of a reference index, a motion vector, and a pixel difference.
For example, in a case of a P picture, a PSKIP mode is widely used. In the PSKIP mode, a reference index is invariably set to 0, a skip vector which is predicted from a motion vector of a peripheral macro block is used as a motion vector, and a pixel difference is set to 0, thus performing encoding. In a case of a B picture, a BSKIP mode is used in a similar manner to the PSKIP mode and further, a direct mode in which only information of a pixel difference is transmitted without transmitting information of a motion vector is used as well.
In the H.264/AVC standard, there is a case where one picture is divided into a plurality of slices so as to conform a defined profile. Examples of such profile in which slice division is defined include a High 10 profile and a High 422 profile. For example, in an example of slice division depicted in FIG. 1, a picture of 1920×1088 pixels is divided into four slices.
When a macro block of 16×16 pixels is used, the picture of FIG. 1 is divided into 68 pieces of horizontal direction block lines and slices 101 to 104 respectively include 17 block lines. Head block lines of the slices 101 to 104 are block lines 111 to 114 respectively. Here, the number of slices in a picture is not limited to four, but the number may be an integer of two or larger and may satisfy a restriction of a profile. Further, slice division may not be performed in a block line unit, but slice division is often performed in a block line unit commonly.
In a case where prediction encoding is performed by using a slice-divided picture as an encoding object, information of another slice is prohibited from being used when prediction is performed by using information of a region which has already been encoded in a certain slice in a picture.
A calculation method of a prediction vector and a skip vector in the H.264/AVC standard is now described with a specific example.
A method for calculating a prediction vector of a P picture is first described. A past reference picture candidate list is called List0 and a future reference picture candidate list is called List1. A reference picture candidate list includes one or more reference indexes which designate a reference picture which is to be a reference candidate. In inter prediction of a P picture, only the List0 is used.
When a reference index refIdx of an encoding object block is determined, a prediction vector mvp of the encoding object block is obtained by using a reference picture which is designated by refIdx. Though a prediction method changes depending on a shape of a sub block, median prediction which is used in a macro block of 16×16 pixels is described here.
In the median prediction, a prediction vector of an encoding object block 201 is obtained by using motion vectors of a macro block A, a macro block B, and a macro block C which are peripheral to the encoding object block 201 and have already been encoded, as depicted in FIG. 2. Reference indexes indicating respective reference pictures of the macro block A, the macro block B, and the macro block C are denoted as refIdxA, refIdxB, and refIdxC respectively. Further, motion vectors of the macro block A, the macro block B, and the macro block C are denoted as mvA, mvB, and mvC respectively. In this case, a prediction vector mvp of an encoding object block is obtained as following.
Step 1: When all of the macro block A, the macro block B, and the macro block C are not available, mvp=0.
Step 2: When the macro block B and the macro block C are not available and the macro block A is available, refIdxB of the macro block B and refIdxC of the macro block C are handled as refIdx equal to refIdxA of the macro block A. Further, mvB of the macro block B and mvC of the macro block C are handled as my equal to mvA of the macro block A. Further, mvA which is a median value of mvA, mvB, and mvC is assigned for mvp.
Step 3: When only one of refIdxA of the macro block A, refIdxB of the macro block B, and refIdxC of the macro block C is equal to refIdx, a motion vector of a macro block which is represented by the reference index is assigned for mvp.
Step 4: When any of step 1 to step 3 does not correspond, a median value of mvA, mvB, and mvC is assigned for mvp.
When prediction encoding is performed by using a slice-divided picture as an encoding object, a macro block existing in the outside of a slice to which the encoding object block 201 belongs is not available because information of another slice is prohibited from being used. Accordingly, when the macro block B and the macro block C belong to another slice, these macro blocks are not available.
A calculation method of a skip vector in the PSKIP mode is now described. 0 is used as a reference index refIdx of an encoding object block. When any of following conditions of still determination is true, both components of a skip vector mvskip which is a motion vector of the encoding object block are set to 0.
(1) The macro block A is not available.
(2) The macro block B is not available.
(3) refIdxA of the macro block A is 0 and both components of mvA are 0.
(4) refIdxB of the macro block B is 0 and both components of mvB are 0.
When any one of the above-mentioned conditions (1) to (4) is not true, a prediction vector is assigned for a skip vector mvskip. Accordingly, when a motion vector of a peripheral macro block is not a zero vector, the corresponding region is not determined as a still region and a non-zero prediction vector is assigned as a skip vector.
In the skip encoding mode, an amount of information used for encoding may be minimized when an encoding object block expresses a motion similar to that of peripheral macro blocks. However, information of another slice is prohibited from being used in the skip encoding mode as well, so that the macro block B is not available when the macro block B belongs to another slice.
Regarding a moving image encoding technique, International Publication Pamphlet No. WO 2007/105590 discloses a method for determining whether or not a skip mode is effective as an encoding mode of a block by using a correction value based on a motion vector of a skip mode.